All sets of initial cliques from methods mClust, boot.sPCA, sPCA, VarCLust and BM (blockmodels) were collected on 200 scale-free graphs, and tested with the VEMtree algorithm. The true clique was also recorded and tested for each seed, and denoted as the oracle.
The questions we adress here are:
All questions are studied with a special regard for the size of the oracle \(nH\) (influence of the missing actor). We define minor, medium and major influence classes as the oracle as follows:
| influence | n |
|---|---|
| Major | 61 |
| Medium | 92 |
| Minor | 47 |
What features has the set of cliques found by each method ?
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 1.9 (0.74) | 30 (14) | 20 (5.3) | 4 (0) | 13 (0) |
| Medium | 2.1 (0.71) | 33 (12) | 21 (5.4) | 4 (0) | 13 (0) |
| Minor | 1.9 (0.72) | 31 (13) | 22 (4.4) | 4 (0) | 13 (0) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.63 (0.28) | 0.43 (0.13) | 0.21 (0.11) | 0.5 (0) | 0.23 (0.036) |
| Medium | 0.56 (0.24) | 0.4 (0.12) | 0.28 (0.079) | 0.5 (0) | 0.27 (0.024) |
| Minor | 0.63 (0.27) | 0.46 (0.1) | 0.3 (0.074) | 0.5 (0) | 0.29 (0.025) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.38 (0.28) | 0.3 (0.2) | 0.65 (0.098) | 0.5 (0) | 0.66 (0.034) |
| Medium | 0.47 (0.25) | 0.46 (0.21) | 0.69 (0.1) | 0.5 (0) | 0.68 (0.024) |
| Minor | 0.38 (0.28) | 0.49 (0.19) | 0.74 (0.11) | 0.5 (0) | 0.69 (0.029) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 8.7 (3.9) | 8.6 (1.9) | 4.3 (0.96) | 7 (0) | 4.3 (0.36) |
| Medium | 7.6 (3.5) | 6.6 (1.5) | 4.1 (0.92) | 7 (0) | 4.1 (0.28) |
| Minor | 8.8 (3.9) | 6.7 (1.1) | 4 (0.73) | 7 (0) | 4.1 (0.28) |
What performances has the set of cliques found by each method ?
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.89 (0.11) | 0.9 (0.12) | 0.77 (0.073) | 0.78 (0.097) | 0.75 (0.043) |
| Medium | 0.85 (0.1) | 0.81 (0.1) | 0.77 (0.049) | 0.78 (0.085) | 0.77 (0.037) |
| Minor | 0.84 (0.08) | 0.79 (0.076) | 0.79 (0.042) | 0.8 (0.048) | 0.8 (0.035) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.86 (0.17) | 0.84 (0.2) | 0.77 (0.15) | 0.68 (0.16) | 0.77 (0.11) |
| Medium | 0.68 (0.23) | 0.6 (0.23) | 0.54 (0.13) | 0.52 (0.18) | 0.59 (0.11) |
| Minor | 0.45 (0.23) | 0.4 (0.2) | 0.38 (0.14) | 0.42 (0.14) | 0.47 (0.14) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.69 (0.27) | 0.75 (0.24) | 0.37 (0.16) | 0.57 (0.16) | 0.35 (0.084) |
| Medium | 0.5 (0.28) | 0.51 (0.25) | 0.27 (0.12) | 0.45 (0.19) | 0.29 (0.08) |
| Minor | 0.41 (0.29) | 0.4 (0.23) | 0.21 (0.11) | 0.42 (0.16) | 0.26 (0.086) |
What feature has the clique corresponding to the maximal value of the lower bound for each method ?
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.56 (0.4) | 0.3 (0.24) | 0.22 (0.23) | 0.37 (0.26) | 0.37 (0.3) |
| Medium | 0.63 (0.31) | 0.44 (0.22) | 0.33 (0.19) | 0.49 (0.22) | 0.47 (0.28) |
| Minor | 0.7 (0.28) | 0.48 (0.17) | 0.34 (0.21) | 0.52 (0.2) | 0.4 (0.33) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.28 (0.27) | 0.2 (0.21) | 0.48 (0.24) | 0.14 (0.15) | 0.23 (0.26) |
| Medium | 0.41 (0.3) | 0.32 (0.32) | 0.55 (0.3) | 0.38 (0.34) | 0.47 (0.37) |
| Minor | 0.38 (0.34) | 0.42 (0.3) | 0.68 (0.27) | 0.39 (0.33) | 0.53 (0.34) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 9.3 (4) | 8.8 (2.3) | 5.9 (1.8) | 9.7 (1.6) | 9 (2.8) |
| Medium | 8.6 (3.6) | 7.8 (2.8) | 5.4 (2.1) | 7.7 (2) | 7 (3.6) |
| Minor | 9.3 (3.9) | 7.2 (2.4) | 4.7 (1.8) | 7.7 (1.8) | 6 (4) |
What is the performance of the clique corresponding to the maximal value of the lower bound for each method ?
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.95 (0.076) | 0.96 (0.11) | 0.89 (0.15) | 0.98 (0.085) | 0.95 (0.12) |
| Medium | 0.88 (0.12) | 0.88 (0.15) | 0.83 (0.15) | 0.84 (0.16) | 0.84 (0.16) |
| Minor | 0.84 (0.09) | 0.81 (0.1) | 0.82 (0.088) | 0.84 (0.11) | 0.84 (0.1) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.94 (0.14) | 0.93 (0.19) | 0.84 (0.28) | 0.95 (0.16) | 0.91 (0.23) |
| Medium | 0.72 (0.28) | 0.73 (0.33) | 0.62 (0.33) | 0.64 (0.33) | 0.62 (0.35) |
| Minor | 0.43 (0.27) | 0.43 (0.28) | 0.42 (0.29) | 0.49 (0.28) | 0.52 (0.29) |
| influence | BM | boot.sPCA | mclust | sPCA | VarClust |
|---|---|---|---|---|---|
| Major | 0.82 (0.24) | 0.9 (0.24) | 0.67 (0.34) | 0.91 (0.19) | 0.85 (0.28) |
| Medium | 0.6 (0.31) | 0.7 (0.37) | 0.5 (0.34) | 0.61 (0.36) | 0.55 (0.38) |
| Minor | 0.43 (0.32) | 0.43 (0.32) | 0.32 (0.25) | 0.54 (0.35) | 0.43 (0.31) |
FN seems to control all performances.
We filter all seeds by their minimum FN reached, for each method and influence.
We filter all seeds by their maximum lower bound value J reached, for each method and influence.
| influence | BM | boot.sPCA | mclust | Oracle | sPCA | VarClust |
|---|---|---|---|---|---|---|
| Major | 0.40 | 0.2 | 0.17 | NA | 0.50 | 0.24 |
| Medium | 0.55 | 0.3 | 0.30 | 1 | 0.48 | 0.28 |
| Minor | 0.70 | 0.4 | 0.36 | 1 | 0.53 | 0.40 |
| influence | BM | boot.sPCA | mclust | Oracle | sPCA | VarClust |
|---|---|---|---|---|---|---|
| Major | 0.65 (0.16) | 0.42 (0.14) | 0.22 (0.15) | NA | 0.52 (0.18) | 0.2 (0.14) |
| Medium | 0.37 (0.23) | 0.42 (0.14) | 0.28 (0.12) | 0 (0) | 0.51 (0.18) | 0.23 (0.11) |
| Minor | 0.4 (0.21) | 0.46 (0.12) | 0.3 (0.094) | 0 (0) | 0.48 (0.14) | 0.27 (0.097) |
| influence | BM | boot.sPCA | mclust | Oracle | sPCA | VarClust |
|---|---|---|---|---|---|---|
| Major | 0.69 (0.23) | 0.55 (0.16) | 0.73 (0.12) | NA | 0.73 (0.23) | 0.78 (0.12) |
| Medium | 0.63 (0.25) | 0.58 (0.19) | 0.73 (0.16) | 0 (0) | 0.58 (0.29) | 0.75 (0.15) |
| Minor | 0.52 (0.25) | 0.5 (0.18) | 0.75 (0.15) | 0 (0) | 0.53 (0.22) | 0.75 (0.13) |
Filtering by failed runs, we observe: